Sleep Quality Estimation using Accelerometer Data from Thigh-Mounted Devices During in Free Living Conditions

(working title)

Esben Lykke, PhD student

03 maj, 2023

Background

Background

  • Sleep plays a vital role in health, thus, improving the assessment of sleep–wake outside of a laboratory environment is critical
  • The gold standard (PSG) is costly and inconvenient.
  • Methods for estimating sleep/wake based on accelerometry exist, primarily from wrist-worn devices
  • Cole-Kripke and Sadeh algorithms are commonly used
  • determine in-bed time is difficult, usually set by sleep log and/or human scorers
  • detect wakefulness is difficult, worse performance in populations with sleep disorders
  • typically a two level analysis: epoch based and summarized across night(s)
  • Zmachine-derived sleep stats
  • Purpose…

But Esben, what about them sleep stages!?

  • I did free-living PSG recordings of sleep but…
    • Super fragile -> shitty data
    • Combersome and time consuming
    • free-living when wired up like a robot?
    • would surface skin temperature + acc be enough? Most likely needs HR

It was likely a dead end from the get-go :(

Methods

Methods

  • data preparation, big time-consumer is handling raw acc data
  • only thigh data used. HSBC and other is only thigh data…
  • all zm recording is considered as in-bed (sensor problem?)
  • no sleep stages, only sleep/awake
  • sensor problems during sleep, up to 20 consecutive epochs (200 sec) are treated as sleep
  • Combine binary classifiers to produce multilabel outcomes (i.e., binary Relevance)
  • Evaluated combined classifiers using weighted micro-averaging.
  • adjust threshold for in bed awake or give edge in tie breakers??
  • zm sleep stats inclusion criteria
  • three approaches: tw- binary (risk of ghost classes), binary relevance (choosing the right tie breaker), and multiclass

Exclusion Criteria

Features

Basic Features

  • Weekday
  • Time of Day
  • Placement
  • Temperature

ACC derived features1

  • Mean ACC X
  • Mean ACC Y
  • Mean ACC Z
  • Standard Deviation X
  • Standard Deviation Y
  • Standard Deviation Z
  • Max Standard Deviation
  • Inclination

Sensor-Independent Features2

  • Clock Proxy Linear
  • Clock Proxy Cosinus

Human Circadian Clock

Forger, Jewett, and Kronauer (1999): a so-called cubic van der Pol equation

\[\frac{dx_c}{dt}=\frac{\pi}{12}\begin{cases}\mu(x_c-\frac{4x^3}{3})-x\begin{bmatrix}(\frac{24}{0.99669\tau_x})^2+kB\end{bmatrix}\end{cases}\]

This thing is dependent on ambient light and body temperature!

Walch et al. (2019) incorporated this feature using step counts from the Apple Watch

But as demonstrated by Walch et al. (2019), a simple cosine function does the trick just as well :)

Circadian Proxy Features

Circadian Proxy Features

Present three modeling approaches

  • Multiclass classifier with three labels: in-bed-asleep, in-bed-awake, and out-bed-awake
    • pros: simplicity, speed, joint modeling, consistency across classes
    • cons: struggle with class imbalance, performance
  • Two binary classifiers: in-bed/out-bed and asleep/awake
    • pros:
    • cons: ghost classes
  • Three binary classifiers: in-bed-asleep 1/0, in-bed-awake 1/0, and out-bed-awake 1/0
    • pros: more flexible, can better handle class imbalance
    • cons: complex training, low speed, feature redundancy

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Estimate Sleep Quality Metrics

Sequential Classifiers

Results

Epoch-Based

  • Performance Metrics
    • F1 Score
    • Accuracy
    • Sensitivity
    • Specificity
    • ROC curves

Summarized across nights

  • Agreement With Zmachine Sleep Stats
    • Sleep Period Time
    • Total Sleep Time
    • Sleep Efficiency
    • Latency Until Persistent Sleep
    • Wake After Sleep Onset
  • Minimal Detectable Change

In-bed models

In-bed performance of decision tree model

Performance Metrics
Grouped by Epoch Length
10 sec 30 sec
F1 Score 93.02% 92.63%
Accuracy 94.25% 93.88%
Sensitivity 93.05% 94.21%
Precision 92.99% 91.10%
Specificity 95.09% 93.65%

Roc curves for sleep models

model epoch_10 epoch_30
raw
Decision Tree 0.79 0.78
Logistic Regression 0.78 0.78
Neural Network 0.81 0.80
XGBoost 0.67 0.70
median_5
Decision Tree 0.73 0.76
Logistic Regression 0.80 0.81
Neural Network 0.82 0.81
XGBoost 0.86 0.82
median_10
Decision Tree 0.74 0.78
Logistic Regression 0.81 0.82
Neural Network 0.84 0.83
XGBoost 0.88 0.83

Performance Metrics
10 second epochs 30 second epochs
Decision Tree Logistic Regression Neural Network XGBoost Decision Tree Logistic Regression Neural Network XGBoost
raw
F1 Score 89.87% 89.26% 89.95% 89.16% 90.03% 89.86% 90.26% 88.73%
Accuracy 83.04% 81.94% 83.31% 82.06% 83.28% 82.74% 83.68% 81.69%
Sensitivity 97.35% 97.19% 96.74% 95.50% 95.70% 96.97% 95.94% 91.48%
Precision 83.45% 82.53% 84.06% 83.61% 84.99% 83.71% 85.22% 86.15%
Specificity 34.45% 30.15% 37.71% 36.44% 37.03% 29.73% 38.00% 45.24%
median_5
F1 Score 91.67% 91.08% 91.91% 92.61% 92.79% 92.78% 93.08% 93.28%
Accuracy 85.53% 84.49% 86.07% 87.33% 87.27% 87.09% 87.80% 88.16%
Sensitivity 98.42% 97.80% 97.79% 98.05% 97.18% 98.36% 97.25% 97.36%
Precision 85.79% 85.22% 86.70% 87.73% 88.79% 87.79% 89.24% 89.52%
Specificity 30.81% 28.03% 36.34% 41.81% 34.06% 26.52% 37.02% 38.75%
median_10
F1 Score 92.26% 91.69% 92.64% 93.38% 92.74% 92.89% 93.27% 93.31%
Accuracy 86.47% 85.45% 87.23% 88.58% 87.16% 87.27% 88.09% 88.20%
Sensitivity 98.31% 97.71% 97.87% 98.11% 96.93% 98.29% 97.51% 97.20%
Precision 86.92% 86.36% 87.94% 89.08% 88.90% 88.06% 89.38% 89.71%
Specificity 32.21% 29.30% 38.49% 44.89% 33.37% 26.63% 36.22% 38.61%

Bland-Altman Plots

::: {.panel-tabset}

https://github.com/esbenlykke/sleep_study

Forger, D. B., M. E. Jewett, and R. E. Kronauer. 1999. “A Simpler Model of the Human Circadian Pacemaker.” Journal of Biological Rhythms 14 (6): 532–37. https://doi.org/10.1177/074873099129000867.
Hirshkowitz, Max, Kaitlyn Whiton, Steven M Albert, Cathy Alessi, Oliviero Bruni, Lydia DonCarlos, Nancy Hazen, et al. 2015. “National Sleep Foundation’s Sleep Time Duration Recommendations: Methodology and Results Summary.” Sleep Health, 4.
Skotte, Jørgen, Mette Korshøj, Jesper Kristiansen, Christiana Hanisch, and Andreas Holtermann. 2014. “Detection of Physical Activity Types Using Triaxial Accelerometers.” Journal of Physical Activity and Health 11 (1): 76–84. https://doi.org/10.1123/jpah.2011-0347.
Walch, Olivia, Yitong Huang, Daniel Forger, and Cathy Goldstein. 2019. “Sleep Stage Prediction with Raw Acceleration and Photoplethysmography Heart Rate Data Derived from a Consumer Wearable Device.” Sleep 42 (12): zsz180. https://doi.org/10.1093/sleep/zsz180.